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DETAILED ACTION 



Specification 



1 . Applicant is reminded of the proper language and format for an abstract of the disclosure. 

The abstract should be in narrative form and generally limited to a single paragraph on a 
separate sheet within the range of 50 to 150 words. It is important that the abstract not exceed 
1 50 words in length since the space provided for the abstract on the computer tape used by the 
printer is limited. 

The abstract is objected to as being too long in length. It is suggested that the appUcant 
revise the abstract to fit within the 50 to 150 word limit. 



The following is a quotation of the appropriate paragraphs of 35 U.S. C. 102 that form the 
basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed pubhcation in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

2. Claims 1-2, 6, 9-12, 14-15, 17-20, 23 and 25-29 are rejected under 35 U.S.C. 102(b) as 
being anticipated by Schlimmer et al. in the article entitled "Quantitative Results Comparing 
Three Intelligent Interfaces for Information Capture: A Case Study Adding Name Information 
into an Electronic Personal Organizer", pubUshed in the December 1996 issue of the Journal of 
Artificial Intelligence Research 5. 



Claim Rejections - 35 USC § 102 
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Referring to claims 1 and 18, Schlinomer et al. teach a method and system comprising 
user input interface code (user interface to convert writing to Unicode text), a field typing engine 
evaluating a program field that has focus against information indicative of whether the field is 
configured to receive text input (deciding if fields such as "Last", "Title", etc, is configured to 
receive user input, i.e. whether the user has taped on the field indicating a desire to enter 
information), and if the field is configured to receive text input (if the user taps on a field), 
providing a visible user input interface at a displayed location relative to the field (the field 
expands to allow users to write information), receiving handwritten data at the input interface, 
providing the handwritten data to a recognition engine, and retuming a recognition result to the 
program (Section 2 on page 330 and 331). This is further shown and explained in Figure 1 . 

Referring to claims 2 and 19, Schlimmer et al. teach the visible user input interface being 
semi-transparent (dotted-Une box representing the expanded field allowing users to write 
information) (Figure 1). 

Referring to claims 6 and 26, Schlimmer et al. teach providing the handwritten data to a 
recognition engine in response to detection of a submit button associated with the visible user 
interface (for example, submitting the name written as a new name by selecting the "New" 
button) (page 331 and further shown in Figure 1). 

Referring to claims 9 and 20, Schlimmer et al. teach evaluating at least one window 
attribute corresponding to the field against retrieved information (for example, opening the menu 
of recently used city names if the user taps on the "City" field) (page 332 and Figure 2). 
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Referring to claim 10, Schlimmer et al. teach accessing window class infonnation (for 
example, opening the menu of recently used city names if the user taps on the "City" field) (page 
332 and Figure 2). 

Referring to claim 11, Schlimmer et al. teach accessing a database to obtain the 
information indicative of whether the field is configured to receive text input (obtaining 
information such as whether the user has tapped a field, such an the "First" field in Figure 1, 
which indicates that the field is ready to receive text input) (page 331 and Figure 1). 

Referring to claim 12, Schlimmer et al. teach adjusting the appearance of the visible input 
window (visible input fields such as the "First" field and "Company" fields can be adjusted, such 
as erasing the input areas by selected the "x" button at the bottom right comer of the input box), 
as shown in Figures 1 and 3. 

Referring to claim 14, Schlimmer et al. teach erasing the visible input window (visible 
input fields such as the "First" field and "Company" fields can be adjusted, such as erasing the 
fields by selected the "x" button at the bottom right comer of the input box), as shown in Figures 
1 and 3. 

Referring to claim 15, Schlimmer et al. teach the visible input window being erased in 
response to receiving a close request (visible input fields such as the "First" field and 
"Company" fields can be adjusted, such as erasing the fields by selected the close request 
represented by the "x" button at the bottom right corner of the input box), as shown in Figures 1 
and 3. 
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Referring to claim 17, Schlimmer et al. teach the visible input window being erased in 
response to a gesture being detected (such as selecting the close request represented by the "x" 
button at the bottom right comer of the input box), as shown in Figures 1 and 3. 

Referring to claim 23, Schlimmer et al. teach the entered data comprising handwritten 
data (page 330), and further comprising a rulebase that determines an appearance of the visible 
input area including a displayed size thereof (for example, changing the size and appearance of 
the input area by closing the input area in response to selection of the "x" button at the bottom 
right comer of the input box) (Figures 1 and 3). 

Referring to claim 25, Schlimmer et al. teach the visible input area has at least one button 
associated therewith for receiving a command (for example, selecting the "New" button for 
performing the command of inputting a new name) (page 33 1 and further shown in Figure 1). 

Referring to claim 27, Schlimmer et al. teach the user input interface code provides the 
recognition result to the program in a message queue associated with the program (for example, 
the "City" field in the Names++ application has a message queue of recently used and 
recognized city names) (page 332 and Figure 2). 

Referring to claim 28, Schlimmer et al. teach the drawing of the visible input area by 
positioning the visible input area relative to the field based on the information received fi'om the 
field-typing engine (expanding the input area based on the field the user taps) (page 33 1 and 
Figure 1). 

Referring to claim 29, Schlimmer et al. teach sizing the visible input area based on the 
information received from the field-typing engine (expanding the input area based on the field 
the user taps) (page 331 and Figure 1). 
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Claim Rejections - 35 USC §103 



The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action; 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

3. Claims 3-5, 7-8, 16, 21-22 and 30-33 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Schlimmer et al. in the article entitled "Quantitative Results Comparing Three 
Intelligent Interfaces for Information Capture: A Case Study Adding Name Information into an 
Electronic Personal Organizer", pubUshed in the December 1996 issue of the Journal of 
Artificial Intelligence Research 5, as appUed to claims 1 and 18 above, and Frink et al. U.S. 
Patent 5,956,423. 

Referring to claim 3, Schlimmer et al. teach all of the limitations as applied to claim 1 
above. However, Schhmmer et al. fail to explicitly teach the handwritten data received at the 
input interface being evaluated to determine whether the handwritten data corresponds to a 
gesture. Frink et al. teach an interface for inputting and recognizing handwritten data (Frink et 
al: column 2, lines 37-65) similar to that of Schlimmer et al. In addition, Frink et al. further 
teach evaluating the handwritten data received at the input interface to determine whether the 
data corresponds to a gesture (Frink et al.: column 8, lines 29-49 and column 10, lines 29-33). It 
would have been obvious to one of ordinary skill in the art, having the teachings of Schlimmer et 
al. and Frink et al. before him at the time the invention was made, to modify the handwriting 
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input interface of Schlimmer et al. to include evaluation of the input for recognition of gestures, 
as taught by Frink et al One would have been motivated to make such a combination in order to 
allow users to easily edit the documents, reducing the confusion of mixing up editing commands 
and data input by the user. 

Referring to claims 4 and 21, Schlimmer et al. teach all of the Umitations as applied to 
claims 1 and 18 above. However, Schhmmer et al. fail to explicitly teach determining when the 
handwritten data corresponds to a gesture and providing at least one pen event corresponding to 
the gesture to the program. Frink et al. teach an interface for inputting and recognizing 
handwritten data (Frink et al: column 2, hues 37-65) similar to that of Schlimmer et al. In 
addition, Frink et al. further teach determining when the handwritten data corresponds to a 
gesture and providing at least one pen event corresponding to the gesture to the program (Frink 
et al: column 2, lines 64-67, column 3, Lines 1-4, column 8, lines 29-49, column 10, lines 63-67 
and column 11, lines 1-4). It would have been obvious to one of ordinary skill in the art, having 
the teachings of Schlimmer et al and Frink et al. before him at the time the invention was made, 
to modify the handwriting input interface of Schlimmer et al. to include evaluation of the input 
for recognition of gestures, as taught by Frink et al. One would have been motivated to make 
such a combination in order to allow users to easily edit the documents, reducing the confusion 
of mixing up editing commands and data input by the user. 

Referring to claims 5 and 22, Schlimmer et al. teach all of the limitations as applied to 
claims 1 and 18 above. Specifically, Schlimmer et al teach a semi-transparent user interface 
(dotted-line box representing the expanded field allowing users to write information) (Schlimmer 
et al.: Figure 1). However, Schlimmer et al. fail to explicitly teach the gesture comprising user 



■ Application/Control Number: 09/976,188 Page 8 

Art Unit: 2173 

input directed to an area of the program that is visible through the user interface, Frink et al. 
teach an interface for inputting and recognizing handwritten data (Frink et al.: column 2, lines 
37-65) similar to that of Schlimmer et al. In addition, Frink et al. further teach users inputting 
gestures directed to an area of the program that is visible through the interface (Frink et al.: 
column 2, lines 64-67 and column 3, lines 52-64 and column). It would have been obvious to 
one of ordinary skill in the art, having the teachings of Schlimmer et al. and Frink et al. before 
him at the time the invention was made, to modify the handwriting input interface of Schlimmer 
et al. to include evaluation of the input for recognition of gestures, as taught by Frink et al. One 
would have been motivated to make such a combination in order to allow users to easily edit the 
documents, reducing the confusion of mixing up editing commands and data input by the user. 

Referring to claim 7, Schlimmer et al. teach all of the limitations as appUed to claim 1 
above. Specifically, Schlimmer et al. teach providing handwritten data to a recognition engine 
(Schlimmer et al.: Section 2 on page 330 and 331 and fixrther shown in Figure 1). However, 
Schlimmer et al. fail to expUcitly teach providing the handwritten data to the recognition engine 
in response to a time being achieved. Frink et al. teach an interface for inputting and recognizing 
handwritten data (Frink et al.: column 2, lines 37-65) similar to that of Schlimmer et al. In 
addition, Frink et al. further teach providing handwritten data to the recognition engine in 
response to a time being achieved (Frink et al.: column 2, lines 53-63). It would have been 
obvious to one of ordinary skill in the art, having the teachings of Schlimmer et al. and Frink et 
al. before him at the time the invention was made, to modify the handwriting input interface and 
recognition engine of Schlimmer et al. to include the recognition of handwritten data in response 
to a time being achieved, as taught by Frink et al. One would have been motivated to make such 



Application/Control Number: 09/976,188 Page 9 

Art Unit: 2173 

a combination in order to allow users to perform functions such as note taking faster and more 
efficiently; recognizing characters after a certain time has elapsed, representing the user has 
completed taking notes, is faster and more efficient than translating the written data character by 
character while the user is taking notes. 

Referring to claim 8, Schlimmer et al. teach all of the limitations as applied to claim 1 
above. Specifically, Schlimmer et al. teach providing handwritten data to a recognition engine 
(Schlimmer et al.: Section 2 on page 330 and 331 and further shown in Figure 1). However, 
Schlimmer et al fail to explicitly teach providing the handwritten data to a recognition engine in 
response to a gesture being detected. Frink et al. teach an interface for inputting and recognizing 
handwritten data (Frink et al.: column 2, hues 37-65) similar to that of Schlimmer et al. In 
addition, Frink et al. further teach providing handwritten data to a recognition engine in response 
to a gesture being detected (Frink et al.: column 10, Unes 29-33). It would have been obvious to 
one of ordinary skill in the art, having the teachings of Schlimmer et al. and Frink et al. before 
him at the time the invention was made, to modify the handwriting input interface of Schlimmer 
et al. to include evaluation of the input for recognition of gestures, as taught by Frink et al. One 
would have been motivated to make such a combination in order to allow users to easily edit the 
documents, reducing the confusion of mixing up editing commands and data input by the user. 

Referring to claim 16, Schlimmer et al. teach all of the limitations as applied to claims 1 
and 14 above. Specifically, Schlimmer et al. teach erasing the visible input window (visible 
input fields such as the "First" field and "Company" fields can be erased by selected the "x" 
button at the bottom right comer of the input box) (Figures 1 and 3). However, Schlimmer et al. 
fail to explicitly point out the input window being erased in response to a time being achieved. 
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Frink et al. teach an interface for inputting and recognizing handwritten data (Frink et al: 
column 2, lines 37-65) similar to that of Schlimmer et al. In addition, Frink et al. further teach 
erasing the input window in response to a time being achieved (sending the handwritten data to 
the recognition engine, and therefore erasing the input, when the user stops writing for a period 
of time) (Frink et al.: column 2, lines 44-63 and Figures 2A, 2B and 2C). It would have been 
obvious to one of ordinary skill in the art, having the teachings of Schlimmer et al. and Frink et 
al. before him at the time the invention was made, to modify the handwriting input interface and 
recognition engine of Schlimmer et al. to include the erasing of the input in response to a time 
being achieved. One would have been motivated to make such a combination in order to allow 
users to perform functions such as note taking faster and more efficiently; recognizing characters 
after a certain time has elapsed, representing the user has completed taking notes, is faster and 
more efficient than translating the written data character by character while the user is taking 
notes. 

Referring to claim 30, Schlimmer et al. teach a system comprising an application 
program having at least one application input area into which user input data can be entered (for 
example, entering name and company information into the Names application), user input 
interface code external to the application program (user interface to convert writing to Unicode 
text), a typing engine that determines whether to call the user interface code for a selected 
application input area of the appUcation program based on attribute information associated with 
that application input area, the user interface code providing a semi-transparent input area based 
on the attribute information when called (deciding if fields such as "Last", "Title", etc. is 
configured to receive user input, i.e. whether the user has taped on the field indicating a desire to 
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enter information; and if the user has tapped on a field, the field expands to allow users to write 
information), and a handwriting recognition engine, configured to receive text information and 
responding by returning recognized text when provided with the information (Schlimmer et al.: 
Section 2 on page 330 and 33 1 and further shown and explained in Figure 1). However, 
Schlimmer et al fail to explicitly teach a timing mechanism and a gesture engine. Frink et al. 
teach an interface for inputting and recognizing handwritten data (Frink et al.: column 2, lines 
37-65) similar to that of Schlimmer et al. In addition, Frink et al. further teach a timing 
mechanism configured to cause removal of the input when no user interaction with the input area 
is detected for a period of time (sending the handwritten data to the recognition engine, and 
therefore erasing the input, when the user stops writing for a period of time) (Frink et al. : column 
2, lines 44-63 and Figures 2A, 2B and 2C), and a gesture engine invoked to determine whether 
the user input data is text or a gesture (Frink et al.: column 8, Unes 29-49 and column 10, lines 
29-33). It would have been obvious to one of ordinary skill in the art, having the teachings of 
Schlimmer et al. and Frink et al. before him at the time the invention was made, to modify the 
input interface and recognition system of Schlimmer et al. to include the timing and gesture 
mechanisms taught by Frink et al. One would have been motivated to make such a combination 
in order to allow users to easily edit the documents, reducing the confusion of mixing up editing 
commands and data input by the user. Furthermore, it allows users to perform functions such as 
note taking faster and more efficiently; recognizing characters after a certain time has elapsed, 
representing the user has completed taking notes, is faster and more efficient than translating the 
written data character by character as the user is taking notes. 
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Referring to claim 31, Schlimmer et al. teach the recognized text is received by the user 
interface code and made available to the application program (the handwriting recognizer 
recognizes the input handwriting, converts it to Unicode text and displays it) (page 330 and 331 
and further shown in Figure 1). 

Referring to claim 32, Schlimmer et al. teach the application program displaying the 
recognized text in the application input area (page 330 and 33 1 and further shown in Figure 1). 

Referring to claim 33, Schlimmer et al. teach a growth rulebase determining whether to 
aher an appearance of the semi-transparent input area in response to the information received 
therein (for example, changing the appearance of the input area by closing the input area in 
response to selection of the "x" button at the bottom right comer of the input box), as shown in 
Figures 1 and 3. 

4. Claims 13 and 24 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Schlimmer et al. in the article entitled "Quantitative Results Comparing Three Intelligent 
Interfaces for Information Capture: A Case Study Adding Name Information into an Electronic 
Personal Organizer", published in the December 1996 issue of the Journal of Artificial 
Intelligence Research 5, as appHed to claims 1, 12, 18 and 23 above, and Microsoft Excel. 

Referring to claims 13 and 24, Schlimmer et al. teach all of the limitations as appHed to 
claims 1, 12, 18 and 23 above. Specifically, they teach adjusting the appearance of the visible 
input window (Figures 1 and 3). However, Schlimmer et al. fail to explicitly teach increasing the 
size of the visible input window based on the data approaching an end thereof and to enable 
entry of additional data. Microsoft Excel (copyright 1999) (see screenshot 1) teaches an input 
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interface that adjusts the appearance of the visible input window (see screenshots 2 and 3 
attached at the end of the office action) similar to that of Schlimmer et al. In addition, Microsoft 
Excel further teaches increasing the size of the visible input window based on the data 
approaching an end thereof and to enable entry of additional data (see screenshots 2 and 3). It 
would have been obvious to one of ordinary skill in the art, having the teachings of Schlimmer et 
al. and Microsoft Excel at the time the invention was made, to modify the input interface of 
Schlimmer et al. to include increasing the size of the input window as needed, taught by 
Microsoft Excel. One would have been motivated to make such a combination in order to allow 
users to enter as much information as needed, making it easier for them to input and view 
information. 

5. The prior art made of record on form PTO-892 and not relied upon is considered 
pertinent to applicant's disclosure. Applicant is required under 37 C.F.R. § 1. 1 1 1(c) to consider 
these references fully when responding to this action. The documents cited therein teach similar 
systems and methods for receiving handwriting input and converting it to recognizable text 
displays. 

Conclusion 

Any inquiry concerning this communication or earlier communications fi'om the 
examiner should be directed to Ting Zhou whose telephone number is (703) 305-0328. The 
examiner can normally be reached on Monday - Friday 8:00 am - 5:30 pm. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Cabeca can be reached on (703) 308-3 1 16. The fax phone number for the 
organization where this application or proceeding is assigned is (703) 872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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